System and method for video communication on mobile devices

ABSTRACT

A video teleconferencing system which allows a mobile device to conduct teleconferences over fixed and mobile packet networks with end points with other mobile devices or a personal computer. The mobile device includes a user interface that displays a video image associated with a first participant. A selectable user interface element corresponding to a second participant video image may be displayed on the user interface and in response to selection of the selectable user interface element, a video image associated with the second participant may be displayed.

CROSS REFERENCE TO RELATED APPLICATIONS

This Application claims the priority of U.S. Provisional Application No.61/303,211 entitled “System and Method for Video Communication on MobileDevices” filed on Feb. 10, 2010, and hereby incorporates by referencethat application.

FIELD OF THE INVENTION

The present invention relates generally to video teleconferencing, andspecifically to methods and systems for video teleconferencing overfixed and mobile packet networks with end points including personalcomputer as well as handheld mobile devices.

BACKGROUND OF THE INVENTION

Video teleconferencing is well known in the art as a means for allowingremote parties to participate in a conversation. Voice, video, andoptionally other data are transmitted between parties over acommunication network, such as the Internet. The parties are able tosee, speak to, and hear the other parties, as well as share other datasuch as a slide presentation.

Being able to conduct video communication on a mobile device isrelatively new development in the art. So called 3G video telephonyservice based on the ITU H.324M standard is the best known practice inthe field. This technology leverages 64 kbps circuit-switchedtransmission channel to send and receive compressed digital audio visualsignals. Due to the inherent limitation of channel bandwidth andprocessing power available on 3G mobile handsets, the technology suffersfrom poor audio and video quality which has prevented the service frombeing widely adopted.

More recent efforts have focused on delivering mobile videocommunications over packet-switched mobile networks. IMS (IntegratedMultimedia Subsystem) is a technology originally devised by the wirelessstandards body 3GPP in an effort to unify the signaling and transmissionof multimedia data including audio and video over the IP (InternetProtocol) layer which in turn can be supported on a variety of lowerlayer networks such as GPRS, Wireless LAN, CDMA 2000, and fixed line.There has been limited deployment of IMS capable mobile networks andstill fewer video communication service offerings that leverage on IMS.AT&T Video Share offers one-way video transmission between two VideoShare capable handsets and is powered by IMS, while relying on theregular cellular voice channel for the audio portion of thecommunication.

BRIEF SUMMARY OF THE INVENTION

The present invention is directed to methods, systems, and software foruse in packet-based video teleconferencing on mobile client devices. Themethods permit mobile client devices to exchange video images and audiodata with other mobile client device as well as personal computerclients in a multipoint conference via a server on the Internet or otherpacket network. Alternatively, point-to-point conferences between twomobile client devices or a mobile client device and a personal computerclient, with or without a dedicated server, are also supported. The coremechanism for conducting such a multipoint or point-to-point videoconference has been previously disclosed in U.S. patent application Ser.No. 11/890,382, the disclosure of which is incorporated herein byreferences. The present invention focuses on the methods, systems, andsoftware that enable mobile devices to participate in such a videoconferencing system. Further, the methods disclosed in the presentinvention can easily be applied by those skilled in the art to enablemobile device participation in other types of video conferencingsystems.

An object of the invention is to minimize battery power consumptionwhile supporting video conferencing usage on mobile devices.

A second object of the invention is to leverage on bandwidth availableto the mobile device provided by multiple underlining networks toachieve enhanced audio and video communication quality of service thatis otherwise not able to be achieved with any single connected network.

A third object of the invention is to maintain continuous connectivityfor the mobile device to participate in a video conferencing sessionwhen the mobile device may be moving in and out of coverage of aparticular network.

A fourth object of the invention is to support multipoint videoconferencing on a mobile device with limited display size.

A fifth object of the invention is to enable the mobile client to beable to conduct video calls with remote parties who are not registeredusers on the same video conferencing system.

To achieve the first object, a system and method is provided whereby themobile video conferencing client software is divided into twocomponents—a background component and a foreground component.Conventional video conferencing client software consists of a singlemachine executable binary program image that is loaded in its entiretyinto the target processor memory space in order to provide the full setof desired functionality. Such practice, when applied on mobile devices,causes high power and memory consumption due to the computationalcomplexity of video conferencing application, and renders unsatisfactorymobile user experience. By separating the client software into twocomponents, a background component and a foreground component, thebackground component can be persistently loaded in processor memory evenduring device's power saving period as its only functionality is tomaintain a light weight connection to the video conferencing server forincoming call notification. When user-driven, UI oriented features arerequired, and the foreground component can then be loaded into processormemory space for the duration that such functionality is needed. Forexample, in one embodiment of the present invention, the backgroundcomponent contains only the minimum functionality of maintaining themobile client's online presence (so other clients can see the mobileclient is up and available for communication) and ability to receive anincoming video call. Upon receiving an incoming video call notification,the foreground component is launched which then provides functionalityfor delivering the full range of services offered by the particularvideo conferencing system.

To achieve the second object, a system and method is provided wherebydata transmission bandwidth available from individually connected mobilenetworks on the mobile device are combined to allow the mobile client toprovide a higher level quality of service for the mobile videoconferencing application. Mobile devices often support connectivity tomultiple networks, such as 3G, 4G, and Wi-Fi networks. In an embodimentof the present invention, the combined bandwidth can be used to send outa higher bit rate video stream to provide higher video image quality. Inanother embodiment of the invention, a portion of the combined availablebandwidth may be used to transmit error protection data packets. Inanother embodiment of the invention, layered video coding technique isused to compress the video signal and different layers of the compressedvideo may be transmitted on different available mobile networks.Characteristics of each connected network channel can be collected andanalyzed to determine the best allocation of the corresponding bandwidthresource for audio, video and/or data transmission during a conferencingsession.

To achieve the third objective, a system and method is provided toenable application layer handover during a video conferencing sessionwhen a mobile client device moves across different types of mobilenetwork coverage, for example, from 4G to 3G, or from cellular networksto WiFi networks. During a network switching event, the audio and videodata transmission can be switched over to the now active channel(s)therefore providing continuous service to the mobile client.

Auxiliary information may be collected and analyzed to predict when suchan application layer handover is necessary to provide an efficienttransition and provide a seamless transition experience. For example, inan exemplary embodiment, the location information reported by the onboard GPS system can be used to check against an up-to-date signalcoverage map to anticipate if the mobile device is moving into or out ofa particular coverage area, or whether a significant change in signalstrength level might occur soon. Such information can be used to preparethe system for hand-over.

Sometimes a mobile device may lose data connection and may be left onlywith a phone connection. In this case the conferencing serverautomatically dials the phone number associated with the mobile clientto allow the mobile device to continue to participate in an ongoingconferencing session. When a data connection is resumed, the mobileclient device reconnects back to the conferencing session and the serverwill subsequently drop the voice connection to the mobile device.

To achieve the fourth object, a system and method is provided to enablea user to view video images from participants in a multipoint videoconferencing session on mobile devices with limited display size.Conventional video conferencing end points typically display videoimages from all participants in a video conferencing session on a singleor sometimes multiple video displays. Such a practice is not possible onmobile devices due to their limited size displays. In an embodiment ofthe present invention, only one video participant's video image isdisplayed on the mobile device at any time during a multi-point videoconferencing session. A pull up menu on the mobile device contains alist of all current participants in the video conferencing session. Eachparticipant is associated with a thumbnail that is periodicallyrefreshed. The user may select any one of the participants to displaythe said participant's full size video stream in the device's maindisplay area. It should be apparent to those skilled in the art that theforegoing is illustrative only and not limiting, having been presentedby way of example only. For example, many other ways of presenting theparticipants' list are possible.

In accordance with the fourth object, a mobile communication devicecomprising a user interface and a computer-readable medium havingexecutable computer-readable program code embodied therein, theexecutable computer-readable program code for implementing a method forcommunication is provided. The method comprises displaying on the userinterface a video image associated with a first participant; displayingon the user interface a selectable user interface element correspondingto a second participant video image; and, displaying on the userinterface a video image associated with the second participant inresponse to selection of the selectable user interface element. Themethod may also comprise the step of displaying on the user interface asecond selectable user interface element where selection of the secondselectable user interface element causes the user interface to displaythe selectable user interface element corresponding to the secondparticipant video image. The video image associated with the firstparticipant is captured at a second mobile communication device and thevideo image associated with the second participant is captured at athird mobile communication device.

The user interface element may be a thumbnail image of a secondparticipant associated with the second participant video image and maybe periodically updated with new images associated with the secondparticipant video image. It is possible that the mobile communicationdevice may receive from a server, video data packets having datacorresponding to the video image associated with a first participant.The mobile communication may decode the received data packets to displayon the user interface the video image associated with a firstparticipant.

In another example, a method for communication between a server one ormore mobile communication devices is disclosed to achieve the fourthobject. The method includes the steps of establishing a communicationlink between a server, a first mobile communication device, a secondcommunication device and a third communication device; transmitting tothe first mobile communication device a video image associated with afirst participant; receiving at the server a request for transmission ofa video image associated with a second participant; and transmitting tothe mobile communication device a video image associated with a secondparticipant in response to receiving at the server a request fortransmission of a video image associated with a second participant. Theserver may receive the video image associated with the first participantfrom the second communication device and receives the video imageassociated with the second participant from the third communicationdevice. The second communication device and the third communicationdevice maybe mobile communication devices or personal computers. Themethod may further comprise the step of stopping transmission to thefirst mobile communication device of a video image associated with afirst participant when the server receives a request for transmission ofa video image associated with a second participant from the first mobilecommunication device

The method may also comprise the steps of displaying on a user interfaceof the first mobile communication device a video image associated with afirst participant; displaying on the user interface of the first mobilecommunication device a selectable user interface element correspondingto the second participant video image; receiving at the first mobilecommunication device a video image associated with a second participantin response to selection of the selectable user interface element; and,displaying on the user interface a video image associated with thesecond participant. The user interface element may be a thumbnail imageof a second participant associated with the second participant videoimage and may be periodically updated with images associated with thesecond participant video image. The method may also include the step ofdisplaying on the user interface of the first mobile communicationdevice a second selectable user interface element wherein selection ofthe second selectable user interface element causes the user interfaceto display the selectable user interface element corresponding to thesecond participant video image

In further accordance with the fourth object, a method for communicationon a mobile communication device is provided. The method includes thesteps of displaying a video image associated with a first participant;displaying a selectable user interface element corresponding to a secondparticipant video image; and, displaying a video image associated withthe second participant in response to selection of the selectable userinterface element. The video image associated with the first participantis captured at a second mobile communication device and the video imageassociated with the second participant is captured at a third mobilecommunication device. The user interface element may be a thumbnailimage of a second participant associated with the second participantvideo image and may be periodically updated with images associated withthe second participant video image.

The mobile communication device may receive from a server, video datapackets having data corresponding to the video image associated with afirst participant and decodes the received data packets to display onthe user interface the video image associated with a first participant.

To achieve the fifth object, a system and method is provided for themobile client to initiate and establish video calls with parties who arenot registered users on the same video communication system as themobile video conferencing client. Traditional video conferencing systemsallow calls to be established between users who are registered on thesame video communication system, via pre-established routing rulesassociated with a specific call signaling protocol. Sometimes a gatewaydevice may be used to route calls across different video communicationsystems, in which case the remote user must be a video communication endpoint that speaks one of the well known video communication protocolsunderstood by the gateway. Such limitations significantly reduce thenumber of potential users a mobile client may be able to reach,presenting a significant barrier for broader adoption of the videocommunication service. The present invention solves this issue byproviding a system and method whereby a mobile client can establishvideo calls with out-of-network users who are reachable via any of theavailable communication means on the mobile device. The initial callinvitation is sent in text format via Email, SMS, IM, or other possiblemechanisms associated with the intended remote party. The invitationmessage contains a URL corresponding to the mobile client's unique webcall-me link assigned by the video conferencing system. Upon receivingthe call invitation message, the receiving party is instructed to clickon the embedded call-me link which will launch a web browser that loadsthe necessary software component and opens a video call session with themobile client.

These and other aspects, objects and embodiments will be more apparentwhen read with the accompanied detailed description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic, pictorial illustration of a system for videoteleconferencing between mobile and PC client devices, in accordancewith an embodiment of the present invention.

FIG. 2A shows a flow chart that schematically represents a method forinitiating video conferencing on the mobile client device by backgroundand foreground components, in accordance with an embodiment of thepresent invention.

FIG. 2B illustrates a schematic diagram of an example of a mobile clientdevice, in accordance with an embodiment of the present invention.

FIG. 3 shows a schematic illustration of a system that bundles threeavailable network data connection channels available to be used in avideo conferencing session, in accordance with an embodiment of thepresent invention.

FIG. 4 shows a flow chart that schematically represents a method forapplication level hand over of a mobile video call, in accordance withan embodiment of the present invention.

FIG. 5 shows a schematic, pictorial illustration of a system to managevideo display during a multi-point video conferencing session, inaccordance with an embodiment of the present invention.

FIG. 6 shows a flow chart that represents a method for establishing avideo communication session between a mobile client device and remoteparties who are not registered on the same video conferencing system asthe mobile client, in accordance with an embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description, numerous specific details are setforth by way of examples in order to provide a thorough understanding ofthe relevant teachings. However, it should be apparent to those skilledin the art that the present teachings may be practiced without suchdetails. In other instances, well known methods, procedures, components,and/or circuitry have been described at a relatively high-level, withoutdetail, in order to avoid unnecessarily obscuring aspects of the presentteachings.

The various technologies described herein generally relate to videoteleconferencing and more specifically to methods and systems for videoconferencing over fixed and packet networks with end points, such aspersonal computers and mobile devices.

FIG. 1. illustrates a high-level schematic of a video teleconferencingsystem 10 for video teleconferencing between multiple devices, such asfor example one or more mobile client devices 11 a, 11 b (generallyreferred to as 11) and one or more personal computer (PC) client devices13. The mobile client devices 11 may send data and communicate over theInternet 15 with other devices 11, 13 via a wireless communicationsnetwork 17. The wireless communications network 17 may be a 3G networkor 4G network, WiFi or any other network protocol or by a combination ofnetworks. In the case of some wireless communications networks, awireless network tower may be used. As shown in FIG. 1 by the lines witharrows, logically each device may communicate with each other by sendingdata to and receiving data from a server 19. In one example, the server19 may be an audio server, a video server, or an audio/video server. Itis also understood that the system may include one or more dedicatedservers, such as a dedicated audio server and a dedicated video server.

FIG. 2A illustrates a flow chart that schematically represents a methodfor initiating video conferencing on the mobile client device 11 bybackground and foreground components. At mobile client device power up21, a mobile client background component is loaded into the mobiledevice's processor memory and starts execution 23. This mobile clientbackground component 25 connects to a communication server over the IPlayer provided by any available data channel of the underlying wirelessnetwork, such as 3G, 4G and WiFi. Preferably, the mobile client device11 needs to possess the necessary credentials on the target videoconferencing system to connect to the communications server. Onceconnected, the mobile client is considered to have successfully loggedonto the video communication service. The server will then notify otherclients connected to the service that this mobile client is availablefor video communication 27. The mobile client sends periodic keep-alivepackets to the communication server to maintain the connection's validopen status 29. The frequency of such keep-alive packets are configuredwith a goal to minimize battery usage, for example, set at every 30 min.However, it is understood that other time periods may also be used.

When a remote client initiates a video call request to the mobileclient, the request is routed via the communication server and sent downto the mobile client via the above described connection maintainedbetween the server and the background component of the mobile client 31.Upon receiving an incoming call message, the background component thenlaunches the foreground component via mechanisms supported by theparticular mobile operating system the mobile device is operated on 33.Once loaded into the processor memory, the foreground component presentsa Graphic User Interface on the mobile device's display screen throughwhich the full functionality of the mobile video conferencing client canbe accessed by the user 33. Upon completion of call handling, theforeground component closes down 35.

FIG. 2B illustrates a schematic diagram of an example of a mobile clientdevice 11. The mobile device may include different schematic layers,including a mobile device hardware layer 37, one ore more device drivers39, a mobile operating system 41, a video conferencing client backgroundcomponent 43 and a video conferencing client foreground components 45.The video conferencing components operate as described above.

Referring now to FIG. 3, the mobile client device 11 may include amobile client user interface 47, a multimedia processing layer 49, and aunified network layer 51. The mobile client user interface layer 47handles interaction with users and allows a user to control theoperation of the client program. It also renders on the device receivedaudiovisual signal of remote video conferencing participants. Themultimedia processing layer 49 manages the capturing and encoding ofaudio and video signals from the device hardware, as well as thedecoding and rendering of audio and video signals received from remoteconference participants. The unified network layer 51 handles thepackaging and transmission of the encoded audio and video data, togetherwith auxiliary information, down to an abstract interface representingthe underlining data networking connection(s). The unified network layer51 also collects data packets received from the underlining datanetworking connections and presents the data up to the media processinglayer as a single logical network interface.

The mobile client device may also include 3G, 4G, and WiFi interfaces53, 55, 57 for interfacing with 3G, 4G, and WiFi networks. Whenestablishing a video communication on a mobile client device 11, themobile client device 11 may sense the type of data connections availableon the mobile client device 11. As noted, the available data connectionsmay include, for example, 3G, 4G, and WiFi connections, If only oneconnection is detected, the mobile client device 11 uses this singledata channel to send and receive audio and video data. If multipleconnections are detected, the mobile client device 11 establishesindependent transmission channels to the video conferencing server oneach of the available communication networks.

The characteristics of the available channels are analyzed to determinehow much bandwidth of each channel is available for audio and videotransmission while meeting a predetermined minimum quality of servicelevel. Net aggregated bandwidth is then reported to the mobile clientapplication layer as the current available transmission bandwidth whichis then used by the mobile client to determine the actual transmissionbit rate of the audio and video signals, as well as which dataconnection will be used for certain data. For example, video data may besent via the WiFi connection, audio data via the 4G network and errorprotection data may be sent via the 3G network.

FIG. 4 illustrates a flow chart that schematically represents a methodfor application level hand over of a mobile video call. A mobile clientdevice 11 is shown connected to multiple data connections and the mobilevideo conferencing client is transmitting audio and video signals of theongoing conferencing over the multiple connections concurrently. Duringa network switch event, one of the underlying networks may becomeunavailable. The mobile client device 11 senses such change and inaccordance will switch to send data originally scheduled for theunavailable channel over other still available channel(s) as well as anynewly available channel(s). The client will adjust the overall datatransmission rate, if necessary, to make the data rate match theavailable bandwidth provided the currently available channels.

For example, with reference to FIG. 4, in a first instance 61, theavailable networks may only include a 4G network. In the first instance,the mobile client device 11 may be transmitting audio and video data ona 4G data connection 55. A first network switch event may occur suchthat in a second instance 63, both a 3G and 4G network connection may beavailable. In the second instance 63, the mobile client device 11 maytransmit different types of data on each of the different networks. Forexample, audio data may be transmitted on the 3G network through the 3Gdata connection 53 and video data may be transmitted on the 4G networkthrough the 4G data connection 55. A second network switch event mayoccur resulting in a third instance 65 in which the 4G network isunavailable and only the 3G network is available. In the third instance,both audio and video data may be transmitted on the 3G network throughthe 3G data connection 53.

FIG. 5 shows a pictorial illustration of a system to manage videodisplay during a multi-point video conferencing session. During amulti-point video conference, the video images of only one of theparticipants 91 may be displayed on the mobile client device 11. In thisexample, other participants' video images are not shown. The respectivevideo streams associated with non-displayed participants may be paused,or may be continuously received. A button, or similar user interfaceelement 93, may be presented on the screen thereby allowing the mobileclient device 11 user to display a new screen that shows thumbnail sizeimages of other participants 95 a-95 d in the video conferencingsession. The thumbnail image may be static, for example, by showing apicture ID associated with the particular party, or dynamically updated.The update frequency may be determined by a predefined refresh rate orwhen there is a triggering event, such as a scene change, detected inthe corresponding video source. The update may be driven by the receiveror the sender side. The user may then select a thumbnail on thethumbnail screen. If the user selects a thumbnail, the mobile clientdevice 11 may elevates the remote party corresponding to the selectedthumbnail to the foreground and displays the full motion and size of thesaid party's video stream. The previous foreground participant replacesthe newly elevated user on the thumbnail screen. In the example shown inFIG. 5, the user may be viewing a single participant 91 on the screen ofthe mobile client device 11. The user may select a user interfaceelement 93 to display a new screen that shows thumbnail size images ofother participants 95 a-95 d in the video conferencing session. The usermay select a thumbnail size image 95 a and in response, the mobileclient device 11 may elevates the remote party corresponding to theselected thumbnail 95 a to the foreground and display the full motionand size of the said party's video stream 97.

FIG. 6 illustrates a flow chart that represents a method forestablishing a video communication session between a mobile clientdevice and remote parties who are not registered on the same videoconferencing system as the mobile client. In step 71, a user may openthe address book on the mobile client device 11 and select an entrycorresponding to the desired remote party with whom the user intended toestablish a video communication session. In step 73, the address booklists the communication channels by which the remote party may bereached. For example, the address book may list a variety of ways toreach a party including the party's phone number, Email address, and oneor several public chat (AOL, MSN, Yahoo, etc.) and social networking(such as facebook) identities. In step 75, if the intended party'saddress book already contains an identity registered on the same videoconferencing system as the mobile video client (i.e., in-network user),a call invitation message is sent following the predefined in-networksignaling protocol and the video call session proceeds, as shown by step77.

If the intended party has not registered on the video conferencingsystem (i.e., out-of-network user) a textual message is sent to any orall of the listed communication identities associated with the remoteparty. For example, the textual message may be sent to the remoteparty's online chat or social network system profile, as shown by step79 or e-mail address, as shown in step 81. For a textual message sent tothe remote party's online chat or social network system profile, thetextual message may be sent in the form of a chat message, as shown instep 83. For a textual message sent to a remote party's e-mail address,the textual message may be in the form of an e-mail as shown in step 85.Alternatively, the textual message may be sent to other communicationidentities of the remote party, such as in the form of an SMS message tothe remote party's mobile client device. A web link is embedded into thetextual message with accompanying instructions to initiate a videoconference. The particular web link is a unique, pre-assigned propertyassociated with each user of the video conferencing system. As shown instep 87, once the textual message is sent to the party, a message issent to the user informing them that a call invitation has been sent tothe party. In the event there are no available communication identitiesassociated with the remote party, a message is sent to the userinforming them that the remote party does not have a valid communicationchannel available, as shown in step 89.

Upon receiving the textual message, the remote party may open thetextual message via one of the available communication channels, followsthe in-message instruction and proceed to click on the embedded linkupon which a web browser window may be opened. Necessary web plug-ins,downloaded if not already present on the personal computer, are thenlaunched within this web video conferencing window and a call invitationmessage is sent to the mobile video client 11. The mobile video clientdevice 11 receives the call notification and proceeds to join the videocall with the remote party.

With reference to FIGS. 1-6, in one example, a first mobile clientdevice 11 a may initiate a video call request to a second mobile clientdevice 11 b. The second mobile client device 11 b may accept the call.The first and second mobile client devices are connected and proceed toexchange encoded real time audiovisual data via an audio video server19. The first mobile client device 11 a receives audiovisual data fromthe second mobile client device 11 b and decodes and displays the videoon the screen of the first mobile client device 11 a. The first mobileclient device 11 a also plays the audio from the second mobile clientdevice 11 b. The second mobile client device 11 b receives audiovisualdata from the first mobile client device 11 a and decodes and displaysthe video on the screen of the second mobile client device 11 b. Thesecond mobile client device 11 b plays the audio from the first mobileclient device 11 a.

The first mobile client device 11 a may initiate a video call request toa third mobile client device (not shown) for joining the ongoing videoconference between the first mobile client device 11 a and the secondmobile client device 11 b. The third mobile client device may accept thecall from the first mobile client device 11 a. The third mobile clientdevice proceeds to connect to the same audio video server 19 that thefirst and second mobile client devices 11 a, 11 b are connected to forexchanging media. The third mobile client may sends its encodedaudiovisual data to the audio video server 19.

The audio video server 19 receives audiovisual data streams from thefirst mobile client device 11 a, the second mobile client device 11 b,and the third mobile client device. The audio video server 19 may decodeall three audio streams and mix “n−1” individually mixed audio streams.The “n−1” individually mixed audio streams is a mix of all audio streamsexcept the one stream generated from the client to which the particularmixed stream is going to be sent. For example, the individually mixedaudio stream sent to the first mobile client device 11 a may includeaudio streams from the second mobile client device 11 b and the thirdmobile client device. Similarly, the individually mixed audio streamsent to the third mobile client device may include audio streams fromthe first mobile client device 11 a and the second mobile client device11 b. The audio video server 19 may then send the individually mixedaudio streams to the corresponding mobile clients. It is preferred thateach mobile client device is able to hear all other mobile clientsduring a multipoint call.

In one exemplary configuration of the system 10, the audio video server19 may send a video data stream generated from the third mobile clientdevice to the first mobile client device 11 a and the second mobileclient device 11 b and send a video stream generated by the first clientmobile device 11 a to the third client mobile device. In thisconfiguration, the first mobile client device 11 a and the second mobileclient device 11 b decode and display video from the third client mobiledevice and the third mobile client device decodes and displays videofrom the first client mobile device 11 a.

In one example of the present system 10, the mobile client device 11 maydisplay a visual cue, such as an icon, to indicate the call now containsmore than 2 participants. The user of the mobile client device 11 may beable to act upon the visual cue, such as clicking or touching theconference icon, to see who is on the call. The user of the mobileclient device 11 may be presented with textual description (such asname), graphic icons (such as avatar), or image thumbnails (such asuser's profile thumbnail) to identify the other participants.

As noted above, a user can select a visual cue on the screen of themobile client device 11 that relates to one of the participants on thecall to see live video from the corresponding participant's mobileclient device 11. For example, in the above configuration, a user onfirst mobile client device 11 a selects a user associated with a secondmobile client device 11 b from the participant list. Upon selection, arequest is sent from first mobile client device 11 a to the audio videoserver 19 which then stops sending video stream generated from thirdmobile client device and starts to send video stream generated from thesecond mobile client device 11 b to the first mobile client device 11 a.The first mobile client device 11 a is now displaying live video fromthe second mobile client device 11 b. The second mobile client device 11a continues to display video from the third mobile client device, andthe third mobile client device continues to display video from the firstmobile client device 11 a.

The audio video server 19 may continuously check to determine if thereis no receiving request for a particular video upstream. If so, theserver 19 may instruct the corresponding client to pause sending itsvideo stream. If the stream is requested again by a mobile client device11, the server 19 may instruct the corresponding client to resumesending its video stream.

In another example, the thumbnails 95 a-95 d representing additionalconference participants are periodically updated with images sent livefrom the corresponding remote participant via the conference audio videoserver 19. The trigger for updating the thumbnail image can be based ontime or other events such as scene change.

In yet another example, the decision as of which participant's video todisplay may be based not on manual selection by user but may be onalgorithmically determined by the audio video server 19 based on aconferencing policy implemented on the audio video server 19. Forexample, it is contemplated that one policy could be to display videoassociated with the current active speaker on all participants' clientdevices. It is contemplated that another policy could be to displayvideo based on selection made by an external conferencing managementmodule (e.g., a moderator selects whose video to display).

It is also understood that in one example of the present system 10, thenumber of live video window on the mobile display can be more than one.For example, on a mobile client device 11 where the display is larger(such as for example a tablet), the system 10 can be configured todisplay more than one live video from remote conference participants,while still representing the rest of participants with thumbnail.

In another example of the system 10, a multipoint video call can containclients with heterogeneous display capability and processing power. Someend points (e.g., a PC client) may be able to display all live videostreams in a call while others (e.g., mobile client) may be limited todisplay one or a subset of video windows.

As shown by the above descriptions, aspects of the systems areimplemented by one or more mobile devices or computers. The hardwareelements, operating systems and programming languages of such devicesare conventional in nature, and it is presumed that those skilled in theart are adequately familiar therewith. In some instances the functionsof the device may be embodied in software, for example programmableinstructions in memory, or hardware, or a combination of hardware andsoftware. For example, the mobile device may have a microprocessor in aportable arrangement, such as, for example, a cellular phone, a smartphone, a personal digital assistant, an audio/video playing device, etc.In another example, the device maybe a PC based implementation of acentral control processing system

In addition, it is understood that the mobile client devices may run avariety of applications programs and store data, enabling one or moreinteractions via the user interface provided, and/or over one or morenetworks to implement the desired processing.

Software, code or a program may take the form of code or executableinstructions for causing a device, or other programmable equipment toperform the relevant data processing steps, where the code orinstructions are carried by or otherwise embodied in a medium readableby a mobile device or other device. Instructions or code forimplementing such operations may be in the form of computer instructionin any form (e.g., source code, object code, interpreted code, etc.)stored in or carried by any readable medium.

It should be noted that various changes and modifications to thepresently preferred embodiments described herein will be apparent tothose skilled in the art. Such changes and modifications may be madewithout departing from the spirit and scope of the present invention andwithout diminishing its attendant advantages.

1. A non-transitory computer-readable medium having executablecomputer-readable program code embodied therein, the executablecomputer-readable program code for implementing a method forcommunication, the method comprising: establishing a video conferenceover a wireless communications network between a first participant usinga mobile communication device and at least one other participant, the atleast one other participant not including the first participant and asecond participant; causing display of, on a user interface of themobile communication device, a video image associated with the firstparticipant; and causing display of, on the user interface of the mobilecommunication device, a first selectable user interface element, whereinselection of the first selectable user interface element during thevideo conference causes the user interface of the mobile communicationdevice to display a second selectable user interface elementcorresponding to the second participant, wherein selection of the secondselectable user interface element during the video conference causes avideo conference request to be transmitted to the second participant,wherein in response to the video conference request being accepted bythe second participant, the user interface of the mobile communicationdevice displays a video image associated with the second participant onthe user interface of the mobile communication device and does notdisplay a video image associated with the first participant or the leastone other participant.
 2. The non-transitory computer-readable medium ofclaim 1 wherein the second selectable user interface element comprises athumbnail image associated with the second participant.
 3. Thenon-transitory computer-readable medium of claim 2 wherein the thumbnailimage of the second participant is periodically updated with imagesassociated with the video image associated with the second participant.4. The non-transitory computer-readable medium of claim 1 wherein themobile communication device receives, from a server, video data packetsdecodes the received data packets to display on the user interface ofthe mobile communication device video images associated with thereceived data packets.
 5. The non-transitory computer-readable medium ofclaim 1 wherein the video image associated with the first participant iscaptured at a second mobile communication device and the video imageassociated with the second participant is captured at a third mobilecommunication device.
 6. The non-transitory computer-readable medium ofclaim 1 wherein the user interface of the mobile communication devicedoes not display the video image associated with the first participantand the least one other participant.
 7. A method for communicationbetween a server and one or more mobile communication devices, themethod comprising: establishing a communication link for a videoconference over a wireless communications network between a server, afirst mobile communication device, a second communication device and athird communication device; transmitting to the first mobilecommunication device participating in the video conference a video imageassociated with a first participant; receiving at the server a requestfor transmission of a video image associated with a second participant;transmitting to the first mobile communication device the video imageassociated with the second participant in response to receiving at theserver the request for transmission of the video image associated withthe second participant; causing a user interface of the first mobilecommunication device to display the video image associated with thefirst participant; causing the user interface of the first mobilecommunication device to display a first selectable user interfaceelement wherein selection of the first selectable user interface elementcauses the user interface of the first mobile communication device todisplay a second selectable user interface element corresponding to avideo image associated with a second participant; transmitting to thefirst mobile communication device the video image associated with thesecond participant in response to selection of the second selectableuser interface element; causing the user interface of the first mobilecommunication device to display the video image associated with thesecond participant, and not display the video image associated with thefirst participant or a third participant.
 8. The method of claim 7wherein the second selectable user interface element is a thumbnailimage of the second participant.
 9. The method of claim 8 wherein thethumbnail image of the second participant is periodically updated withimages associated with the second participant.
 10. The method of claim 7wherein the server receives the video image associated with the firstparticipant from the second communication device and receives the videoimage associated with the second participant from the thirdcommunication device.
 11. The method of claim 7 wherein the secondcommunication device and the third communication device are mobilecommunication devices.
 12. The method of claim 7 further comprising thestep of stopping transmission to the first mobile communication deviceof the video image associated with the first participant when the serverreceives a request for transmission of the video image associated withthe second participant from the first mobile communication device. 13.The method of claim 7 wherein the wireless communications networkcomprises at least one of 3G, 4G, and Wi-Fi networks.
 14. Anon-transitory computer-readable medium having executablecomputer-readable program code embodied therein, the executablecomputer-readable program code for implementing a method forcommunication, the method comprising: establishing a video conferenceover a wireless communications network between a first participant usinga mobile communication device and at least one other participant, the atleast one other participant not including the first participant and asecond participant; displaying on a user interface of the mobilecommunication device a video image associated with the firstparticipant; and displaying on the user interface of the mobilecommunication device a first selectable user interface element whereinselection of the first selectable user interface element causes the userinterface of the mobile communication device to display a secondselectable user interface element corresponding to the secondparticipant; wherein selection of the second selectable user interfaceelement causes a video conference request to be transmitted to thesecond participant, wherein in response to the video conference requestbeing accepted by the second participant, the user interface of themobile communication device displays a video image associated with thesecond participant on the user interface of the mobile communicationdevice and does not display a video image associated with the firstparticipant or the at least one other participant.
 15. Thenon-transitory computer-readable medium of claim 14 wherein the secondselectable user interface element is a thumbnail image of the secondparticipant.
 16. The non-transitory computer-readable medium of claim 15wherein the thumbnail image of the second participant is periodicallyupdated with images associated with the second participant video imageassociated with the second participant.
 17. The non-transitorycomputer-readable medium of claim 14 wherein a server receives the videoimage associated with the first participant from the communicationdevice and receives the video image associated with the secondparticipant from the third communication device.
 18. The non-transitorycomputer-readable medium of claim 14 wherein the user interface of themobile communication device does not display the video image associatedwith the first participant and the least one other participant.
 19. Thenon-transitory computer-readable medium of claim 14 wherein the wirelesscommunications network comprises at least one of 3G, 4G, and Wi-Finetworks.
 20. The non-transitory computer-readable medium of claim 14wherein establishing a video conference over a wireless communicationsnetwork comprises establishing a video conference over at least twowireless communications networks.